A binaural audio signal is a stereo signal made up of the left and right signals reaching the left and right ear drums of a listener in a real or virtual 3D environment. Streaming or playing a binaural signal for a person through a good pair of headphones allows the listener to experience the immersive sensation of being inside the real or virtual environment, because the binaural signal contains all of the spatial cues for creating that sensation.
In real environments, binaural signals are recorded using small microphones that are placed inside the ear canals of a real person or an artificial head that is constructed to be acoustically equivalent to that of the average person. One application of streaming or playing such a binaural signal for another person through headphones is to enable that person to experience a performance or concert almost as “being there.”
In virtual environments, binaural signals are simulated using mathematical modeling of the acoustic waves reaching the listener's eardrums from the different sound sources in the listener's environment. This approach is often referred to as 3D audio rendering technology and can be used in a variety of entertainment and business applications. For example, gaming represents a significant commercial application of 3D audio technology. Game creators build immersive 3D audio experiences into their games for enhanced “being there” realism.
However, use of 3D audio rendering technology goes well beyond gaming. Commercial audio and video conferencing systems may employ 3D audio processing in an attempt to preserve spatial cues in conferencing audio. Further, many types of home entertainment systems use 3D audio processing to simulate surround sound effects, and it is expected that new commercial applications of 3D environments (virtual worlds for shopping, business, etc.) will more fully use 3D audio processing to enhance the virtual experience.
Conventionally, the reproduction of reasonably convincing sound fields, with accurate spatial cueing, during playback of 3D audio relies on significant signal processing capabilities, such as those found in gaming PCs and home theater receivers. (References to “3D audio” in this document can be understood as referring specifically to binaural audio with its discrete left and right ear channels, and more generally to any audio intended to create a spatially-cued sound field for a listener.)
Delivery of a binaural signal to a listener through headphones is straightforward, because the left binaural signal is delivered directly to the listener's left ear and the right binaural signal is delivered directly to the listener's right ear. However, the use of headphones is sometimes inconvenient and they isolate the listener from the surrounding acoustical environment. In many situations that isolation can be restricting. Because of those disadvantages, there is great interest in being able to deliver binaural and other 3D audio to listeners using a pair of external loudspeakers.
To appreciate the difficulty involved in delivering such audio, FIG. 1 illustrates an overall loudspeaker transmission system 10 from two loudspeakers 12L and 12R to the eardrums 14L and 14R of a listener 16. The diagram depicts the natural filtering of the loudspeaker signals SL and SR on their way to the listener's left and right ear drums 14L and 14R. The sound wave signal SL from the left speaker 12L is filtered by the ipsilateral head related (HR) filter HI(ω) before reaching the left ear drum 14L and by the contralateral HR filter HC(ω) before reaching the right ear drum 14R. Corresponding filtering occurs for the right loudspeaker signal SR.
The main problem with the illustrated signal transmission system 10 is that there are crosstalk signals from the left loudspeaker to the right ear and from the right loudspeaker to the left ear. As a further problem, the HR filtering of the direct term signals by the ipsilateral filters HI(ω) colors the spectrum of the direct term signals. The equations below provide a complete description of the left and right ear signals in terms of the left and right loudspeaker signals:
                                          E            L                    ⁡                      (            ω            )                          =                                                            H                I                            ⁡                              (                ω                )                                      ⁢                                          S                L                            ⁡                              (                ω                )                                              +                                                                                                                                                                  H                          C                                                ⁡                                                  (                          ω                          )                                                                    ⁢                                                                        S                          R                                                ⁡                                                  (                          ω                          )                                                                                      ,                                    ︸                                                                                                      Crosstalk                  ⁢                                                                                                    ⁢                                                                                                  ⁢                  right                                                                                                      speaker                  ⁢                                                                          ⁢                  to                  ⁢                                                                          ⁢                  left                  ⁢                                                                          ⁢                  ear                                                                                        Eq        .                                  ⁢                  (          1          )                                and                                                                                          E              R                        ⁡                          (              ω              )                                =                                                                                                                                                                  H                          C                                                ⁡                                                  (                          ω                          )                                                                    ⁢                                                                        S                          L                                                ⁡                                                  (                          ω                          )                                                                                      ︸                                                                                                                    Crosstalk                    ⁢                                                                                  ⁢                    right                                                                                                                    speaker                    ⁢                                                                                  ⁢                    to                    ⁢                                                                                                              ⁢                                                                                                            ⁢                                          lef                      ⁢                      t                                        ⁢                                                                                  ⁢                    ear                                                                        +                                                            H                  I                                ⁡                                  (                  ω                  )                                            ⁢                                                S                  R                                ⁡                                  (                  ω                  )                                                                    ,                            Eq        .                                  ⁢                  (          2          )                    where EL and ER are the left and right ear signals, respectively, and SL and SR are the left and right loudspeaker signals, respectively.
If a left binaural signal BL was transmitted directly from the left speaker 12L and a right binaural signal BR was transmitted directly from the right speaker 12R, the signals at the listener's ears would be given byEL(ω)=HI(ω)BL(ω)+HC(ω)BR(ω),  Eq. (3)andER(ω)=HC(ω)BL(ω)+HI(ω)BR(ω).  Eq. (4)These actual left and right ear signals are much different from the desired left and right ear signals, which areEL(ω)=e−jωτBL(ω),  Eq. (5)andER(ω)=e−jωτBR(ω).  Eq. (6)Where τ is a given, system-dependent time delay.
In Eq. (3) and Eq. (4), the spatial audio information originally present in the binaural signals is partly destroyed by the head related filtering of the direct-path terms. However, the main degradation is caused by the crosstalk signals. With crosstalk, the signals reaching each of the listener's ears are a mix of both the left and right binaural signals. That mixing of left and right binaural signals completely destroys the perceived spatial audio scene for the listener.
However, the desired left/right ear signals as given in Eq. (5) and Eq. (6) can be obtained, or nearly so, by filtering and mixing the binaural signals before transmission by the loudspeakers 12L and 12R to the listener 16. FIG. 2 illustrates a known approach to filtering and mixing binaural signals in advance of loudspeaker transmission, providing the listener 16 with left/right ear signals more closely matching the desired left/right ear signals.
In the diagram, a prefilter and mixing block 20 precedes the loudspeakers 12L and 12R. The illustrated prefiltering and mixing block 20 is often called a crosstalk cancellation block and is well known in the literature. It includes a left-to-left direct-path filter 22L and a right-to-right direct-path filter 22R. Each direct-path filter 22 implements a direct-term filtering function denoted as PD. The block further includes a left-to-right cross-path filter 24L and a right-to-left cross-path filter 24R. Each cross-path filter 24 implements a cross-path filtering function denoted as PX.
With these prefilters and their illustrated interconnections, a left-path combiner 26L mixes the left direct-path signal together with the right-to-left cross-path signal, and the right-path combiner 26R mixes the right direct-path signal together with the left-to-right cross-path signal. From the diagram, it is easily seen that the left ear signal EL is given by:
                                                                                                              E                    L                                    ⁡                                      (                    ω                    )                                                  =                                ⁢                                                                            H                      I                                        ⁡                                          (                      ω                      )                                                        ⁢                                                            S                      L                                        ⁡                                          (                      ω                      )                                                        ⁢                                                            S                      R                                        ⁡                                          (                      ω                      )                                                                                                                                              =                                ⁢                                                                                                    H                        I                                            ⁡                                              (                        ω                        )                                                              ⁢                                          (                                                                                                                                  P                              D                                                        ⁡                                                          (                              ω                              )                                                                                ⁢                                                                                    B                              L                                                        ⁡                                                          (                              ω                              )                                                                                                      +                                                                                                            P                              X                                                        ⁡                                                          (                              ω                              )                                                                                ⁢                                                                                    B                              R                                                        ⁡                                                          (                              ω                              )                                                                                                                          )                                                        +                                                                                    H                        C                                            ⁡                                              (                        ω                        )                                                              ⁢                                          (                                                                                                                                  P                              X                                                        ⁡                                                          (                              ω                              )                                                                                ⁢                                                                                    B                              L                                                        ⁡                                                          (                              ω                              )                                                                                                      +                                                                                                                                                                                    ⁢                                                                            P                      D                                        ⁡                                          (                      ω                      )                                                        ⁢                                                            B                      R                                        ⁡                                          (                      ω                      )                                                                      )                                                    ⁢                                  ⁢                                  ⁢                                                                                                  =                                                                  (                                                                                                                                            H                                I                                                            ⁡                                                              (                                ω                                )                                                                                      ⁢                                                                                          P                                D                                                            ⁡                                                              (                                ω                                )                                                                                                              +                                                                                                                    H                                C                                                            ⁡                                                              (                                ω                                )                                                                                      ⁢                                                                                          P                                X                                                            ⁡                                                              (                                ω                                )                                                                                                                                    )                                            ︸                                                                                                                                                              R                      D                                        ⁡                                          (                      ω                      )                                                                                            ⁢                                          B                L                            ⁡                              (                ω                )                                              +                                          ⁢                                          ⁢                                                                                                                (                                                                                                                                  H                              I                                                        ⁡                                                          (                              ω                              )                                                                                ⁢                                                                                    P                              X                                                        ⁡                                                          (                              ω                              )                                                                                                      +                                                                                                            H                              C                                                        ⁡                                                          (                              ω                              )                                                                                ⁢                                                                                    P                              D                                                        ⁡                                                          (                              ω                              )                                                                                                                          )                                        ︸                                                                                                                                          R                      X                                        ⁡                                          (                      ω                      )                                                                                            ⁢                                                            B                  R                                ⁡                                  (                  ω                  )                                            .                                                          Eq        .                                  ⁢                  (          7          )                    Symmetric results are obtained for the right ear signal ER.
To obtain the desired binaural signal transmissions specified in Eq. (5) and Eq. (6), the direct-path transfer function RD(ω) from BL to EL needs to satisfy:RD(ω)=HI(ω)PD(ω)+HC(ω)PX(ω)=e−jωτ,  Eq. (8)and the cross-path transfer function RX(ω) from BR to EL must satisfy:RX(ω)=HI(ω)PX(ω)+HC(ω)PD(ω)=0.  Eq. (9)Eq. (8) and Eq. (9) can be used to obtain a general purpose solution for the direct-path filter PD and the cross-path filter PX. Such solutions are well known in the literature, but their implementation requires relatively sophisticated signal processing circuitry.
In an increasingly mobile world, however, more and more audio playback occurs on devices that have limited signal processing capabilities and great sensitivity to overall power consumption. Perhaps more significantly, such devices commonly have fixed speakers that generally are very closely spaced together (e.g., 30 cm or less). For example, mobile terminals, computer audio systems (especially for laptops/palmtops), and many teleconferencing systems use loudspeakers positioned within close proximity to each other. Because of their limited processing capabilities and their close speaker spacing, the recreation of spatial audio by such devices is particularly challenging.